Pandas Profiling Report for Nhabe Radar dataset

Dataset statistics

Number of variables	8
Number of observations	694500
Missing cells	2637265
Missing cells (%)	47.5%
Duplicate rows	180
Duplicate rows (%)	< 0.1%
Total size in memory	42.4 MiB
Average record size in memory	64.0 B

Variable types

Numeric	8

Alerts

Dataset has 180 (< 0.1%) duplicate rows	Duplicates
`reflectivity` is highly overall correlated with `total_power`	High correlation
`total_power` is highly overall correlated with `reflectivity`	High correlation
`reflectivity` has 669781 (96.4%) missing values	Missing
`total_power` has 591788 (85.2%) missing values	Missing
`velocity` has 690519 (99.4%) missing values	Missing
`spectrum_width` has 685177 (98.7%) missing values	Missing

Reproduction

Analysis started	2024-04-18 09:32:34.661037
Analysis finished	2024-04-18 09:32:41.600306
Duration	6.94 seconds
Software version	ydata-profiling vv4.7.0
Download configuration	config.json

longitude
Real number (ℝ)

Distinct	628805
Distinct (%)	90.5%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%
Mean	10.696635

Minimum	7.9691935
Maximum	13.349942
Zeros	0
Zeros (%)	0.0%
Negative	0
Negative (%)	0.0%
Memory size	5.3 MiB

Quantile statistics

Minimum	7.9691935
5-th percentile	8.7690589
Q1	10.026549
median	10.681468
Q3	11.388629
95-th percentile	12.592085
Maximum	13.349942
Range	5.3807485
Interquartile range (IQR)	1.3620805

Descriptive statistics

Standard deviation	1.0907156
Coefficient of variation (CV)	0.1019681
Kurtosis	-0.2611624
Mean	10.696635
Median Absolute Deviation (MAD)	0.681308
Skewness	-0.031300086
Sum	7428812.9
Variance	1.1896604
Monotonicity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
10.65961	1390	0.2%
10.659763	11	< 0.1%
10.659426	8	< 0.1%
10.6580925	8	< 0.1%
10.659678	7	< 0.1%
10.659652	7	< 0.1%
10.6553545	6	< 0.1%
10.680629	6	< 0.1%
10.659812	6	< 0.1%
10.657205	6	< 0.1%
Other values (628795)	693045	99.8%

Minimum 10 values
Maximum 10 values

Value	Count	Frequency (%)
7.9691935	1	< 0.1%
7.969471	1	< 0.1%
7.9695344	1	< 0.1%
7.9705777	1	< 0.1%
7.9706645	1	< 0.1%
7.9719095	1	< 0.1%
7.9722543	1	< 0.1%
7.972479	1	< 0.1%
7.972832	1	< 0.1%
7.972857	1	< 0.1%

Value	Count	Frequency (%)
13.349942	1	< 0.1%
13.349633	1	< 0.1%
13.349595	1	< 0.1%
13.348596	1	< 0.1%
13.348468	1	< 0.1%
13.3476095	1	< 0.1%
13.347404	1	< 0.1%
13.347104	1	< 0.1%
13.346597	1	< 0.1%
13.346461	1	< 0.1%

latitude
Real number (ℝ)

Distinct	388402
Distinct (%)	55.9%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%
Mean	106.75614

Minimum	103.99065
Maximum	109.46598
Zeros	0
Zeros (%)	0.0%
Negative	0
Negative (%)	0.0%
Memory size	5.3 MiB

Quantile statistics

Minimum	103.99065
5-th percentile	104.76983
Q1	106.05684
median	106.75115
Q3	107.47215
95-th percentile	108.69708
Maximum	109.46598
Range	5.475326
Interquartile range (IQR)	1.4153063

Descriptive statistics

Standard deviation	1.122769
Coefficient of variation (CV)	0.010517137
Kurtosis	-0.30164581
Mean	106.75614
Median Absolute Deviation (MAD)	0.7081075
Skewness	-0.04248564
Sum	74142142
Variance	1.2606102
Monotonicity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
106.728325	1389	0.2%
106.73351	20	< 0.1%
106.73381	19	< 0.1%
106.732925	18	< 0.1%
106.72284	18	< 0.1%
106.72406	17	< 0.1%
106.7338	17	< 0.1%
106.73006	16	< 0.1%
106.72719	16	< 0.1%
106.72699	16	< 0.1%
Other values (388392)	692954	99.8%

Minimum 10 values
Maximum 10 values

Value	Count	Frequency (%)
103.990654	1	< 0.1%
103.99074	1	< 0.1%
103.99149	1	< 0.1%
103.991646	1	< 0.1%
103.993126	1	< 0.1%
103.993324	1	< 0.1%
103.99335	1	< 0.1%
103.99342	1	< 0.1%
103.99409	1	< 0.1%
103.99422	1	< 0.1%

Value	Count	Frequency (%)
109.46598	1	< 0.1%
109.46589	1	< 0.1%
109.46525	1	< 0.1%
109.46504	1	< 0.1%
109.46374	1	< 0.1%
109.46367	1	< 0.1%
109.46317	1	< 0.1%
109.46315	1	< 0.1%
109.462395	1	< 0.1%
109.46235	1	< 0.1%

altitude
Real number (ℝ)

Distinct	23167
Distinct (%)	3.3%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%
Mean	7281.3861

Minimum	10
Maximum	24684
Zeros	0
Zeros (%)	0.0%
Negative	0
Negative (%)	0.0%
Memory size	5.3 MiB

Quantile statistics

Minimum	10
5-th percentile	390
Q1	2482
median	5940
Q3	10989
95-th percentile	18700
Maximum	24684
Range	24674
Interquartile range (IQR)	8507

Descriptive statistics

Standard deviation	5807.8991
Coefficient of variation (CV)	0.79763647
Kurtosis	-0.056046087
Mean	7281.3861
Median Absolute Deviation (MAD)	3961
Skewness	0.84620343
Sum	5.0569227 × 10⁹
Variance	33731692
Monotonicity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
10	1389	0.2%
26	389	0.1%
15	256	< 0.1%
48	243	< 0.1%
43	227	< 0.1%
36	206	< 0.1%
86	203	< 0.1%
87	201	< 0.1%
125	185	< 0.1%
60	183	< 0.1%
Other values (23157)	691018	99.5%

Minimum 10 values
Maximum 10 values

Value	Count	Frequency (%)
10	1389	0.2%
13	1	< 0.1%
14	18	< 0.1%
15	256	< 0.1%
16	33	< 0.1%
17	2	< 0.1%
18	7	< 0.1%
19	7	< 0.1%
20	69	< 0.1%
21	177	< 0.1%

Value	Count	Frequency (%)
24684	1	< 0.1%
24670	1	< 0.1%
24641	2	< 0.1%
24627	3	< 0.1%
24624	1	< 0.1%
24612	1	< 0.1%
24609	1	< 0.1%
24598	3	< 0.1%
24584	1	< 0.1%
24581	2	< 0.1%

reflectivity
Real number (ℝ)

HIGH CORRELATION MISSING

Distinct	5382
Distinct (%)	21.8%
Missing	669781
Missing (%)	96.4%
Infinite	0
Infinite (%)	0.0%
Mean	1.5609632

Minimum	-35.57
Maximum	57.89
Zeros	7
Zeros (%)	< 0.1%
Negative	13015
Negative (%)	1.9%
Memory size	5.3 MiB

Quantile statistics

Minimum	-35.57
5-th percentile	-15.75
Q1	-8.86
median	-1.15
Q3	11.65
95-th percentile	24.991
Maximum	57.89
Range	93.46
Interquartile range (IQR)	20.51

Descriptive statistics

Standard deviation	13.135602
Coefficient of variation (CV)	8.4150616
Kurtosis	-0.42415256
Mean	1.5609632
Median Absolute Deviation (MAD)	9.4
Skewness	0.51791294
Sum	38585.45
Variance	172.54403
Monotonicity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
-7.85	23	< 0.1%
-6.51	22	< 0.1%
-8.04	20	< 0.1%
-10.75	19	< 0.1%
-9.8	19	< 0.1%
-6.22	18	< 0.1%
-9.19	17	< 0.1%
-5.51	17	< 0.1%
-7.99	17	< 0.1%
-10.5	17	< 0.1%
Other values (5372)	24530	3.5%
(Missing)	669781	96.4%

Minimum 10 values
Maximum 10 values

Value	Count	Frequency (%)
-35.57	1	< 0.1%
-34.47	1	< 0.1%
-33.63	1	< 0.1%
-32.73	1	< 0.1%
-32.18	1	< 0.1%
-31.36	1	< 0.1%
-31.23	1	< 0.1%
-31.19	1	< 0.1%
-31.09	1	< 0.1%
-30.84	1	< 0.1%

Value	Count	Frequency (%)
57.89	1	< 0.1%
55.61	1	< 0.1%
54.36	1	< 0.1%
52.3	1	< 0.1%
51.79	1	< 0.1%
50.37	1	< 0.1%
50.24	1	< 0.1%
48.15	1	< 0.1%
47.7	1	< 0.1%
46.94	1	< 0.1%

total_power
Real number (ℝ)

HIGH CORRELATION MISSING

Distinct	7857
Distinct (%)	7.6%
Missing	591788
Missing (%)	85.2%
Infinite	0
Infinite (%)	0.0%
Mean	8.6153701

Minimum	-25.62
Maximum	77.68
Zeros	30
Zeros (%)	< 0.1%
Negative	33295
Negative (%)	4.8%
Memory size	5.3 MiB

Quantile statistics

Minimum	-25.62
5-th percentile	-10.82
Q1	-2.68
median	6.77
Q3	16.83
95-th percentile	37.26
Maximum	77.68
Range	103.3
Interquartile range (IQR)	19.51

Descriptive statistics

Standard deviation	14.847889
Coefficient of variation (CV)	1.7234186
Kurtosis	1.1415539
Mean	8.6153701
Median Absolute Deviation (MAD)	9.69
Skewness	0.96230355
Sum	884901.89
Variance	220.45981
Monotonicity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
17.11	102	< 0.1%
17.13	99	< 0.1%
17.1	91	< 0.1%
17.12	88	< 0.1%
17.07	80	< 0.1%
17.09	75	< 0.1%
17.08	74	< 0.1%
17.05	70	< 0.1%
17.04	70	< 0.1%
17.14	70	< 0.1%
Other values (7847)	101893	14.7%
(Missing)	591788	85.2%

Minimum 10 values
Maximum 10 values

Value	Count	Frequency (%)
-25.62	1	< 0.1%
-24.73	1	< 0.1%
-23.81	1	< 0.1%
-23.41	1	< 0.1%
-23.26	1	< 0.1%
-22.44	1	< 0.1%
-22.16	1	< 0.1%
-21.37	1	< 0.1%
-21.36	1	< 0.1%
-20.99	1	< 0.1%

Value	Count	Frequency (%)
77.68	1	< 0.1%
76.35	1	< 0.1%
74.9	1	< 0.1%
74.67	1	< 0.1%
74.16	1	< 0.1%
74.13	1	< 0.1%
73.04	1	< 0.1%
72.81	1	< 0.1%
72.38	1	< 0.1%
72.29	1	< 0.1%

velocity
Real number (ℝ)

MISSING

Distinct	786
Distinct (%)	19.7%
Missing	690519
Missing (%)	99.4%
Infinite	0
Infinite (%)	0.0%
Mean	-0.30993469

Minimum	-4.02
Maximum	4.02
Zeros	2
Zeros (%)	< 0.1%
Negative	2285
Negative (%)	0.3%
Memory size	5.3 MiB

Quantile statistics

Minimum	-4.02
5-th percentile	-3.82
Q1	-2.62
median	-0.89
Q3	2.15
95-th percentile	3.81
Maximum	4.02
Range	8.04
Interquartile range (IQR)	4.77

Descriptive statistics

Standard deviation	2.6060645
Coefficient of variation (CV)	-8.4084311
Kurtosis	-1.3590743
Mean	-0.30993469
Median Absolute Deviation (MAD)	2.24
Skewness	0.24840312
Sum	-1233.85
Variance	6.7915721
Monotonicity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
-3.77	19	< 0.1%
-3.83	15	< 0.1%
-3.85	15	< 0.1%
-3.23	15	< 0.1%
-2.17	14	< 0.1%
3.99	14	< 0.1%
-3.84	14	< 0.1%
-3.74	13	< 0.1%
-3.76	13	< 0.1%
-3.82	13	< 0.1%
Other values (776)	3836	0.6%
(Missing)	690519	99.4%

Minimum 10 values
Maximum 10 values

Value	Count	Frequency (%)
-4.02	1	< 0.1%
-4.01	7	< 0.1%
-4	12	< 0.1%
-3.99	10	< 0.1%
-3.98	10	< 0.1%
-3.97	10	< 0.1%
-3.96	11	< 0.1%
-3.95	10	< 0.1%
-3.94	6	< 0.1%
-3.93	9	< 0.1%

Value	Count	Frequency (%)
4.02	4	< 0.1%
4.01	11	< 0.1%
4	10	< 0.1%
3.99	14	< 0.1%
3.98	9	< 0.1%
3.97	9	< 0.1%
3.96	9	< 0.1%
3.95	13	< 0.1%
3.94	12	< 0.1%
3.93	9	< 0.1%

spectrum_width
Real number (ℝ)

MISSING

Distinct	227
Distinct (%)	2.4%
Missing	685177
Missing (%)	98.7%
Infinite	0
Infinite (%)	0.0%
Mean	0.98428081

Minimum	0.01
Maximum	2.52
Zeros	0
Zeros (%)	0.0%
Negative	0
Negative (%)	0.0%
Memory size	5.3 MiB

Quantile statistics

Minimum	0.01
5-th percentile	0.29
Q1	0.74
median	0.99
Q3	1.23
95-th percentile	1.65
Maximum	2.52
Range	2.51
Interquartile range (IQR)	0.49

Descriptive statistics

Standard deviation	0.39548235
Coefficient of variation (CV)	0.4017983
Kurtosis	0.1303397
Mean	0.98428081
Median Absolute Deviation (MAD)	0.24
Skewness	0.046976429
Sum	9176.45
Variance	0.15640629
Monotonicity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
0.97	125	< 0.1%
1.07	121	< 0.1%
0.96	118	< 0.1%
0.95	117	< 0.1%
0.99	115	< 0.1%
1.02	114	< 0.1%
1.03	114	< 0.1%
0.93	113	< 0.1%
1.09	110	< 0.1%
0.91	109	< 0.1%
Other values (217)	8167	1.2%
(Missing)	685177	98.7%

Minimum 10 values
Maximum 10 values

Value	Count	Frequency (%)
0.01	96	< 0.1%
0.02	1	< 0.1%
0.04	4	< 0.1%
0.05	3	< 0.1%
0.06	4	< 0.1%
0.07	7	< 0.1%
0.08	5	< 0.1%
0.09	5	< 0.1%
0.1	5	< 0.1%
0.11	4	< 0.1%

Value	Count	Frequency (%)
2.52	1	< 0.1%
2.43	1	< 0.1%
2.39	1	< 0.1%
2.37	1	< 0.1%
2.36	3	< 0.1%
2.3	1	< 0.1%
2.29	1	< 0.1%
2.23	2	< 0.1%
2.22	1	< 0.1%
2.2	2	< 0.1%

time
Real number (ℝ)

Distinct	159
Distinct (%)	< 0.1%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%
Mean	87.973362

Minimum	1
Maximum	171
Zeros	0
Zeros (%)	0.0%
Negative	0
Negative (%)	0.0%
Memory size	5.3 MiB

Quantile statistics

Minimum	1
5-th percentile	8
Q1	48
median	91
Q3	132
95-th percentile	163
Maximum	171
Range	170
Interquartile range (IQR)	84

Descriptive statistics

Standard deviation	49.782457
Coefficient of variation (CV)	0.56588103
Kurtosis	-1.1833595
Mean	87.973362
Median Absolute Deviation (MAD)	42
Skewness	-0.075074601
Sum	61097500
Variance	2478.293
Monotonicity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
93	5500	0.8%
166	5500	0.8%
79	5500	0.8%
46	5000	0.7%
133	5000	0.7%
64	5000	0.7%
4	5000	0.7%
101	5000	0.7%
151	5000	0.7%
159	5000	0.7%
Other values (149)	643000	92.6%

Minimum 10 values
Maximum 10 values

Value	Count	Frequency (%)
1	3500	0.5%
2	4500	0.6%
3	4500	0.6%
4	5000	0.7%
5	4500	0.6%
6	4000	0.6%
7	4000	0.6%
8	5000	0.7%
9	4000	0.6%
10	4500	0.6%

Value	Count	Frequency (%)
171	4000	0.6%
170	4000	0.6%
169	4500	0.6%
168	4500	0.6%
167	4000	0.6%
166	5500	0.8%
165	4500	0.6%
164	3500	0.5%
163	4000	0.6%
162	5000	0.7%

Auto

Heatmap
Table

	altitude	latitude	longitude	reflectivity	spectrum_width	time	total_power	velocity
altitude	1.000	-0.003	-0.016	0.322	-0.061	0.456	-0.289	-0.066
latitude	-0.003	1.000	-0.037	0.195	0.017	-0.153	0.004	-0.111
longitude	-0.016	-0.037	1.000	-0.219	0.225	-0.154	0.019	0.057
reflectivity	0.322	0.195	-0.219	1.000	0.142	-0.468	0.889	-0.065
spectrum_width	-0.061	0.017	0.225	0.142	1.000	-0.117	0.365	-0.009
time	0.456	-0.153	-0.154	-0.468	-0.117	1.000	-0.280	0.078
total_power	-0.289	0.004	0.019	0.889	0.365	-0.280	1.000	-0.024
velocity	-0.066	-0.111	0.057	-0.065	-0.009	0.078	-0.024	1.000

Count
Matrix

A simple visualization of nullity by column.

Nullity matrix is a data-dense display which lets you quickly visually pick out patterns in data completion.

First rows
Last rows

	longitude	latitude	altitude	reflectivity	total_power	velocity	spectrum_width	time
0	10.659610	106.728325	10.0	NaN	23.16	NaN	NaN	9.0
1	10.665006	106.728360	15.0	NaN	33.11	NaN	NaN	9.0
2	10.670401	106.728390	21.0	8.91	16.82	NaN	NaN	9.0
3	10.675796	106.728424	27.0	18.73	25.21	NaN	NaN	9.0
4	10.681191	106.728460	34.0	19.04	31.96	NaN	NaN	9.0
5	10.686587	106.728490	40.0	NaN	39.84	NaN	NaN	9.0
6	10.691982	106.728530	46.0	NaN	42.31	NaN	NaN	9.0
7	10.697378	106.728560	52.0	NaN	57.06	NaN	NaN	9.0
8	10.702774	106.728600	58.0	NaN	59.59	NaN	NaN	9.0
9	10.708170	106.728630	65.0	NaN	59.56	NaN	NaN	9.0

	longitude	latitude	altitude	reflectivity	total_power	velocity	spectrum_width	time
694490	13.291256	106.683620	23543.0	NaN	NaN	NaN	NaN	131.0
694491	13.296610	106.683525	23601.0	NaN	NaN	NaN	NaN	131.0
694492	13.301966	106.683430	23660.0	NaN	NaN	NaN	NaN	131.0
694493	13.307321	106.683340	23718.0	NaN	NaN	NaN	NaN	131.0
694494	13.312674	106.683240	23777.0	NaN	NaN	NaN	NaN	131.0
694495	13.318028	106.683160	23835.0	NaN	NaN	NaN	NaN	131.0
694496	13.323383	106.683060	23894.0	NaN	NaN	NaN	NaN	131.0
694497	13.328738	106.682976	23952.0	NaN	NaN	NaN	NaN	131.0
694498	13.334093	106.682880	24011.0	NaN	NaN	NaN	NaN	131.0
694499	13.339448	106.682785	24069.0	NaN	NaN	NaN	NaN	131.0

Most frequently occurring

	longitude	latitude	altitude	reflectivity	total_power	velocity	spectrum_width	time	# duplicates
42	10.65961	106.728325	10.0	NaN	17.04	NaN	NaN	143.0	4
64	10.65961	106.728325	10.0	NaN	17.07	NaN	NaN	142.0	4
126	10.65961	106.728325	10.0	NaN	17.12	NaN	NaN	126.0	4
16	10.65961	106.728325	10.0	NaN	16.97	NaN	NaN	159.0	3
35	10.65961	106.728325	10.0	NaN	17.03	NaN	NaN	30.0	3
37	10.65961	106.728325	10.0	NaN	17.03	NaN	NaN	161.0	3
44	10.65961	106.728325	10.0	NaN	17.04	NaN	NaN	154.0	3
47	10.65961	106.728325	10.0	NaN	17.05	NaN	NaN	23.0	3
50	10.65961	106.728325	10.0	NaN	17.06	NaN	NaN	20.0	3
58	10.65961	106.728325	10.0	NaN	17.06	NaN	NaN	146.0	3

Overview

Variables

Interactions

Correlations

Missing values

Sample

Duplicate rows

Most frequently occurring